Woman overboard

Why can't I just learn to say no!?
Last night at work, a guy came in I hadn't seen in a while. We... Well, for lack of a better term, were "fuck buddies" when I was in high school. He was with another woman, but while she was busy at the counter, he came back and talked to me...Flirting, hinting at things that we had done "back then", and asked for my phone number and email... And I gave it to him!
I know for a fact that he is not interested in what I am looking for... I want a real relationship, I want someone that respects me, cares about me. He just wants someone to "play" with... I KNOW THAT, so WHY did I not just say no??!?
When I married my husband, I loved him. But I also had, in the back of my mind, the thought that no one else would ever ask me to marry him, so I'd better say yes. That recording seems to be still playing in my head, as if I should feel grateful to any man who shows the least bit of attention to me. I know that is what I am doing, yet when the man is standing there I don't think about it... I don't say no... I simply grab onto the attention like a woman overboard grabbing a flotation device.  I get so mad at myself sometimes... I seem to be scared to say no...
It's the same thing with the man I mentioned in an earlier post, Stuart. All he is looking for is a good time... But I tell him, oh yeah, I'll call.... Instead of just telling him that want different things, and a "roll in the hay" is not what I want...
Someday, somehow, this has got to change... But IM not sure how to get there...